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DESCRIPTION AND RATIONALE 



OF THE TRAINING CONSORTIUM 

In this technical paper, the rationale for the 
training consor t ium is given, as well as a more com- 
prehensive description of each of the consortium units. 
Included latoi in this technical paper is a discussion 
of the multiple roles that will be filled by each con- 
sortium unit. 

R a tio n ale fo r « he Con so r ti urn 

Consortiums are a lot of work. They are difficult 
to establish and even more difficult to maintain. Para- 
phrasing the well known statement r "Two persons can keep 
a secret if one of them is dead," one might conclude that 
two organizations can form a consortium if one of them 
is aoout to lose its funding. Successful consortiums 
require that somehow, within a rather loosely-drawn 
confederation, each constituent unit can find rewards 
from a common endeavor. 

Of course, consortiums once established and operated 
enjoy many advantages and alternative options simply 
not available io any of the organizations operating singly* 
In the particular case under consideration here, the 
rationale for the inclusion of the consortium units, and 
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the multiple values that thus resulted are broken down into 
values during the planning of the training program, and 
values expected during the actual conduct of the training. 

A . Value of Consortium Agencies during P lanning 

During planning, the principal value of the consortium 
agencies seemed to be concentrated in three areas. First, 
and obvious, consortium units served as sources of informa- 
tion and provided data that served as valuable planning 
input. Agencies were generous in the amount of time and 
consideration that they gave to the several issues involved 
in initiating the training consortium. To encourage each 
constituent unit to independently think through relevant 
issues in terms of the organization's own best interests, 
the following procedure was established. Early in the plan- 
ning, a key member of each agency was asked to write a work- 
ing paper considering sequentially a series of questions and 
topics that appeared relevant for designing the training 
program. A format for these working papers was provided 
and is included as Appendix A in Technical Paper Number 1. 
Seven of the consortium units found the necessary manpower 
and time to develop a working paper even though there were 
severe time constraints operating on them. These seven 
papers have been included as Appendices A through G in 
this second technical paper. Included are working papers 
by two national curriculum projects fBioloqical Sciences 
Curriculum Study and Earth Sciences Educational Program) , 
by two regional laboratories (Southwestern Cooperative 
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Educational Laboratory and Southwest Regional Laboratory 
for Educational Research and Development) , by two agencies 
representing the public schools (the Denver Public Schools 
and the Northern Colorado Educational Board of Cooperative 
Services) , and by the Colorado Department of Education. 

These working papers have been edited slightly in the 
interest of consistency of style; in some cases, iDOsitions 
of consortium units have changed somewhat since the working 
p ape r s wc re d r a fled. 

It. should be stressed that consortium units wre asked 
to serve, in a sense, the organization's own self interest 
in developing the working papers. Additionally, no attempt 
was made to overstructure or unduly influence consort ium 
agencies as to the particular content that they felt 
should be in their working papers. For those c jencies who 
ecu] d not find sufficient time to respond in writing, indi- 
vidual planning sessions were held so that each agency had 
an opportunity to express its feelings on the several issues 
at hand. 

A second rationale for including consortium units during 
the planning phase was because of their willingness and ex- 
pertise as eritiquers of planned and/or proposed training. 
Just as the plans of the consortium benefited from the sev- 
eral independent views that were expressed on the operation 
of consortiums generally, they also profited from the multi- 
ple opinions and views that were brought to bear upon the 
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proposed train inn program. This criticism occurred 
throughout the design grant period, culminating in 
November as each consortium unit reacted to and influenced 
the content of the Preliminary Final Report. In some 
cases, activities from mid-November to mid-December, 1970, 
resulted in many additional alterations to the Preliminary 
Final Report. The vast differences in orientation of the 
consortium units, which w i 1 i become apparent as the reader 
scans the description of the several agencies, virtually 
insured that a number of conflicts of o t least moderate 
intensity would arise while determining the specifics of 
the planned training. Nevertheless, there was opportunity 
tc resolve most: differences to the satisfaction of the 
agencies involved. 

A third obvious value of the consortium units during 
planning is closely related to the second value mentioned 
above. Fiecuuse of their unique characteristics, each 
agency has specific values and orientations quite different 
from the values and orientations of the other units. The 
rich mix and exchange of ideas that this consortium triggered 
assisted materially in enhancing, we believe, the feasibility 
of the proposal lie re in contained. 

B . Vn 1 ue of C o nsortium Agencies during T raining 

Moving from the planning phase to the contemplated 
operational or training phase, the rationale for inclusion 
of consortium units expand and take cu equal, if not greater 




G 



Closely related to 



importance, than during planning, 
one point above (that is, serving as critiquers of planned 
training), the several member units will also function 
as direct and valuable critiquers of the actual training 
that takes place. This is as it should be. In many cases, 
as noted below, consortium units will be in a very direct 
position tc observe and otherwise evaluate the training 
that is being conducted as well as the trainees that are 
being produced by the program. 

A second rationale for the use of consortium units 
during the training is that most of them serve as valuable 
sources of trainees for the various programs. fhis is 
obvious in the case of the public schools, but it is also 
true of nearly all the consortium units. They either have 
on board employees who would benefit from additional high 
quality training in specific areas or they would like to 
hire personnel trained in educational evaluation or 
development. Additionally, and related to the point in 
the proceeding paragraph, when these consortium units 
provide personnel from their organizations with the opportu 
nity to take pari in a training program, it becomes 
logical for them to evaluate the training that the person 
receives by his performance when he returns to the parent 
organ! za t? on. 

A third value in that the agencies servo as training 
sites themselves. Because of other constraints on their 
resources (particularly their manpower) , certain of these 
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agencies will not function as a training site per se . 

That is to say, they du not have sufficient personnel to 
permit them to conduct formal courses of training. 

Certain development projects, however, do have on board 
a staff with the competencies and the necessary time to 
devote to such undertakings; it appears that several 
effective training sites actually will be at development 
projects (see Technical Paper Number 3, Section IV). 

A fourth value that readily can be seen is that 
the consortium units serve as a potential source of 
instructors for the various training programs. This can 
be visualized most easily in terms of the proposed 
intensive training institutes. For example, staff at CIRCE, 
staff at the Regional Laboratories, staff at the Northern 
Colorado Educational Board of Cooperative Services, staff 
at the John F. Kennedy Child Development Center, staff 
at the national curriculum projects, and staff at nearly 
all of the consortium units provide a viable and varied 
manpower pool from which to dra*-* in developing a cadre 
of instructors. This is probably particularly true, as 
noted above, for the shorter training institutes. This 
ability to draw upon expertise from within the consortium 
itself, where the goals of the training program are agreed 
upon and well known, certainly should benefit the training 
program. It is also apparent that the strength and 
effectiveness of the institutes likely can be enhanced by 
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the inclusion of ftarf from outside the consortium units; 
especially staff of high competency who arc able to devote 
more than a few days to a training institute. 

Each of the consortium jc nicies wil?. se.ve as an 
internship site. The extort of involve; .on* v. i J 1 hr dependent 
upon many things, among tnem, the partxc . Lc. ,ueus of the 
internship site and the particular desire? u. 1 mpetencies 
of the trainee. The variety of internship s • to s available 
is pronounced; its richness reflects the versatility of the 
consoitium units. Thus, internship sites e • s t at CIRCE 
in innovative approaches to evaluai.ion ol curriculum and 
of other projects. Strong, viable internships in development 
easily can be established at the national curr: cuium projects, 
Active internships, likewise, can be structured with regional 
laboratories? it might be noted that the Southwest Regional. 
Laboratory (SWRL) has established an internship program that 
is entering its fourth year. The internships at the John F. 
Kennedy Child Development Center cut across many high priority 
areas, such as minority groups, early childhood, reading 
difficulties of children, and the like. Extensive 
opportunities exist for internships with public school 
districts where the evaluation trainee would likely be 
cast in a decision-oriented mode, while the trainee in 
development could become involved in curri cuium development 
underway in the school districts. Internships 'with the 
State Department of Education, with the Ford Foundation, or 
with the Regional Office of the Office of Education could 
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provide valuable experience for interns in conducting 
evaluations of programs of considerable scope and wide 
s igni f 1 cance . 

A final reason for including consortium units 
during Lhe actual training is identical to one of the 
reasons for including them during the planning of the 
training. Simply stated, the diversity of their views 
enhances the effectiveness, efficiency, and feasibility 
of the training program. Their unique, personalistic 
perspectives guarantee that a broad spectrum of opinions 
will be sampled as critical decision points ere reached 
in the training program. 
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II. Description of Consortium Units 

Each of the consortium units is considered briefly 
and simply below; space does not permit elaborating upon 
all of the ramifications of each of the institutions' 
activities. Nevertheless , an attempt is made to provide 
sufficient information that the reader will have a better 
understanding of the essential characteristics of each 
of the agencies. 

1 ) T he University of Colorad o 

The University of Colorado, with its campuses at 
Boulder, Colorado Springs, and Denver, is increasing 
rapidly in size and in stature as a center for 
excellence and innovation in education. It was the 
site of much of the early work in "the new mathematics/* 
and in recent years has attracted to Boulder the High 
School Geography Project* the Biological Science 
Curriculum Study Committee, the Earth Science Curriculum 
Project, as well ac the Social. Science Education 
Consortium (these other consortium units are discussed 
below) . As an example of an unique program recently 
established, consider the Center for Education in 
the Social Sciences; it is designed to promote 
communica tion , innovation, und research directed 
toward high quality pre-service and in-service 
education of teachers in the social sciences. The 
Center is a cooperative effort between the social science 
departments and the School of Education. 




* Terminated August 31, 1970. 
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The University's library system is exceptional in 
many respects. Among other high points, it is 
excellent with respecu to its Education Library , 

Documents Library, and Automation and Systems Division. 

The director of the Automation and Systems Division 
is an expert on systems design and computer applications. 
An American College and Research Libraries Association 
survey, 1969, showed that the University of Colorado 
Education Library is surpassed, in size and significance, 
caly by the education libraries of Harvard and 
Stanford Universities. The library takes a vigorous 
approach to making contacts and supplying services 
both inside and outside the University, including work 
for the Education Commission of the States, which is 
located in Denver. The library maintains a file on 
United States and Canadian education dissertations, 
on which an annual summary and analysis has been completed 
and published (by Phi Delta Kappa) each year. 

The Univeisity of Colorado has an excellent Bureau 
of Audiovisual Instruction. Its staff has had 
extensive experience in teacher-education activities 
and in the creation of all kinds of audiovisual material. 
The bureau will serve as an excellent resource during 
some phases of development training. 

The University of Colorado houses two major computer 
centers operated by expert technical personnel. One 
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on the main campus employs the IBM 360, Model 40, 

128 K. The machine is partitioned in four areas 
with two areas foreground, and two areas background 
for telecommuni cation purposes. The version is F 
release of COBOL. Machine configuration 4, 9 channel, 
800 BPI tape drive; three 23-11 disc drives; one card 
reader punch? and one 1403 N-l printer. The Scientific 
Center of the Graduate School is located on East Campus 
and uses the CDC 6400, The operating system is 
SCOPE, 3.1, 6 version. The machine has six high-speed 
tape drives, one high-speed printer, and is capable c:‘ 
multiple-processing. It is capable of 60,000 words 
core storage, equivalent extended storage, and over- 
one million words of disc storage. The machine has 
telecommunication capability. While these facilities 
have been continously pressed to match growing 
University demands, the facilities and services will 
be available to the trainees in evaluation and develop- 
ment. 

Also housed within the University and more directly 
relevant to this specific proposal is the Laboratory 
of Educational research with its Tit^e IV, ESEA, 
graduate training program in research. The laboratory 
has built many bridges to diverse agencies while 
establishing an active program of internship and 
service. During the academic year 1968-69, for 
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example, the total group of research trainees 
(twelve in number) perfor?ned over 5,000 hours of 
internship activities while extending services 
to over forty agencies ? n the region. This disciple- 
like activity has continued unabated and, in fact, 
increased over the years. The Laboratory has 
established itself as one of the primary sites for 
research training in the United Steiles. It is fully 
expected th£*t multiple interfaces will be possible 
between the Title IV research trainees in the 
laboratory, and the trainees undergoing the evaluation 
and development training program. 

2 ) The Center for Instructional Re se arch and Curriculum 
Eva laution (CIRCE) 

The Center for Instructional Research and Curriculum 
Evaluation housed within the University of Illinois, 
Urbana campus, has established itself as a leading 
national site in evaluation theorizing and implementa- 
tion. Its rise to prominence in the last half of the 
decade of the 1960‘s '^fleets a remarkable ability 
to operate with the parctitioner on the one hand and 
with the more theoretically oriented, University-based 
colleague, on the other. 

The objectives that have been predominant for CIRCE 
have been: 

(1) To furnish help to curriculum projects by 
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evaluating curiicula, instructional materials, and 
methods of instruction. 

(2) To develop new concepts of relationships between 
learning and measurement and to study the dissemination 
of innovations; to consolidate research results 
across fields and levels of learning. 

(3) To disseminate development products (tests, 
methods, models) to professionals, schools, colleges, 
and interested organizations in order to improve 
educational practices. 

(4) To increase the output of well- trained researchers, 
evaluators, and developers, and to assist in the 
reorientation of trained behavioral science researchers 
of other fields toward educational concerns. 

In their concern with the operational and conceptual 
problems of curriculum projects, CIRCE has indeed 
been fortunate to be surrounded by several national 
curriculum projects of tremendous import. For example, 
the Curriculum Laboratory established at the University 
of Illinois is performing several vital functions. It 
houses several groups, such as a curriculum development 
jroup for each of the disciplines, a group experimenting 
with laboratory practices os part of the undergraduate 
preparation of teachers, a group experimenting with 
methods for teaching gifted children, a group dealing 
with exceptional children, and the like. An experimental 
school is part of the Laboratory as well as other labora- 
tories, shops, and supporting facilities. The Curriculum 
Laboratory also has research, training, and service 
functions. Additional projects (other than the 
Curriculum Laboratory) include the University of Illinois 
Committee of School Mathematics, the Demonstration 
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Project for Gifted Youth, the English Curriculum 
Project, and several programs focusing on the cognitive 
development of pre-school children. 

The national reputation of CIRCE and its ongoing 
activities make it ideal as an internship site for 
doctoral students in evaluation and development. 

Exposure of the doctoral candidate to more than one 
orientation (e.g., Colorado) is seen as desirable. 

3 ) Biological Sc ie nces Curriculum Study (BSCS ) 

The Biological Sciences Curriculum Study commenced 
its activities in the late 1950*^ and has established 
itself as a leading factor in changing the content of 
science courses in the nation’s schools; in fact, the 
basic materials produced by BSCS also have been trans- 
lated into many different foreign languages. Its 
support has been drawn primarily from the National 
Science Foundation and its continuous funding reflects 
the high quality output that it has produced. For 
example, one unique project of BSCS, funded by the 
U.S. Office of Education’s Bureau of the Handicapped, 
includes the preparation of courses in the life sciences 
for educable mentally handicapped children. From 
inception of the project in the Summer of 1969, 

BSCS staff have moved rapidly, field testing their 
first package with eleven to thirteen year olds in 
Spring, 1970. The materials are exciting and unique; 
for example, reading by the student is not necessary. 
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Opportunities for stimulating internships in develop- 
ment and, in part, evainat’on are found within BSCS ' s 
many activities. The BSCS working paper (Appendix A) 
contains well thought-out procedures for enhancing training 
in development. 

4 ) Earth Science Educational Program (ESEP) 

The Eaich Science Educational Program, like BSCS, is 
located in Boulder and is one of the group of curriculum 
reform projects supported by NSF. Work on ESEP began in 
1963 under the sponsorship of the American Geological 
Institute. ESEP includes three related sub-projects: Earth 

Science Curriculum Project (ESCP); Environmental Studies (ES) 
and Earth Science Teacher Preparation Project (ESTPP) . The 
major objective of the project is the improvement of earth 
science education in secondary schools. Very close contacts 
have teen maintained between the BSCS and ESEP staff since 
the beginning of the earth science curriculum revision. 

T l curriculum materials developed by ESEP have reflected 
an orientation towards directed discovery and an emphasis 
on pupil acquisition of behaviors. In attempts to strengthen 
their curriculum approach/ the project's staff has recently 
been attempting to build into their materials the intrinsic 
motivation model, even though this means moving away from 
a rigidly-prescribed curriculum. ESEP has moved toward 
the creation o) flexible learning environments and has 
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increasingly recognized the paramount importance of 
teacher training . It exists as a vital and active 
site for development interns, as well as for cogent 
evaluation activities (see Appendix B) . 

5) High School Geography Project 

Although the High School Geography Project was 
phased out in August, 1970, the experiences in 
curriculum development that they have had over the past 
several years provided an important resource for 
planners of the design project. Since the geography 
project has teen disbanded, it cannot serve as a site 
for interns or for other training. Still it was felt 
appropriate to acknowledge, in this proposal, the 
assistance on development that was forthcoming from 
the staff of that project. 

6 ) Social Sciences Education Cons o rtium (SSEC) 

Also located in Boulder, the Social Sciences Education 
Consortium is providing leadership in the area of the 
social sciences as are BSCS and FJSCP in their fields- 
Incorporated in 1965 as a non-profit corporation, 
the original consortium was founded in 1963 at a 
Purdue University conference. In August, 1967, the 
main offices moved to Boulder, Colorado, to the 
University of Colorado campus. Support for the work 
of the SSEC has come from several sources such us NSF , 
the U.S. Office of Education, and the Charles F. 
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Kettering Foundation. f fhe consortium is composed 
of sixty outstanding social scientists and educators 
who have been active in developing and implementing 
new ideas and new materials in the social sciences. 

Its many activites related to curriculum development 
and evaluation of social science materials make it 
an excellent site for internsnips and training. 

7) S outhwestern Coope ra tiv e E duca tional Laboratory (SWCKL) 
Located in Albuquerque, New Mexico, the Southwestern 
Cooperative Educational Laboratory is one of the group 
of laboratories funded under Title IV of the Elementary 
and Secondary Education Act of 1965. Its major program 
interest is to develop and improve first year school 
experience in the languerge arts with initial focus on 
~al language tor Chicane, Indian, and black children. 
Programs are being developed to improve the p re -school 
acquisition of oral language, to continue oral language 
instruction in the primary grades, and to ease the 
transition from oral language to reading. The many 
activities underway at SWCEL make it an opportune 
location for interns in both development and evaluation 
(see Append i x C) . 

3 ) Southwest Regional Laboratory for Educational Research 
and Development (SWRL ) 

The Southwest Regional Laboratory for Educational 
Research and Development is located in Inglewood, 
California. Major program foci for SWRL are the 
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following: development of a coordinated primary 

grade curriculum that includes communication skills, 
problem solving, and humanities elements; development 
of a computer-managed instructional system to aid the 
teacher, and a computer-based planning system to assist 
the school administrator in decision making; and develop- 
ment in instructional materials to train school per- 
sonnel who use SWI1L developed products. These programs 
result in four primary areas of development: in 

commun i cation skills for grades K through 4; in 
generalized problem solving skills for grades K through 4; 
in computer-managed instruction in reading, loading 
readiness, and mathematics at the first grade level; 
and in the computer -managed administrative planning 
system to assist the administrator in decision making. 

SWKL has cooperated with the Systems Development 
Corporation in Santa Monica, California, on the 
development of the computer programs. This advanced 
orientation to development (see Appendix D) marks 
SWRL as an excellent site for interns, again in both 
evaluation and development . 

J ) . John F. Kenne dy Chil d Dev e lopme n t center - University of 
Colorado Medical Center 

The John F. Kennedy Child Development Center presents 
unique opportunities for internships, primarily in 
evaluation but also in development. '[he range of 
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activities underway at the center is broad; an 
expert resident professional staff conducts various 
training programs as well as works with children in 
numerous settings. 

Three projects currently unde i way are funded under 
the Education Professions Development Act and provide 
the nucleus from which exceptional internship opportunities 
could emerge. The firs program ii evolves a demonstration 
pre-school laboratory ui.at has been established as an 
integral part of the development center. This project, 
titled the Early Childhood Special Educutjon Fellowship 
Program, is designed to train early childhood special 
educators to deal with exceptional children in the 
regular pre-school program or regular primary class- 
room. It serves eight experienced teachers (full 
academic year fellowships) plus ten tc ucher - trai ners 
and five college preservice teacher -trai ners (one 
month practiouins) who have agreed to collaborate in 
initiating new early childhood programs or changing 
existing programs . Trainees participate in all 
activities of the Center, which is an interdisciplinary 
evaluation and treatment program for children who have 
various developmen 1 al disabilities* All trainees learn 
about the complexities of normal and abnormal child 
development from many points of view. The TP DA trainees 
focus on designing and imp leu ent i ng individualized 
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educational programs loi' chiicinui ^Lih ii;;oc i f i. e 
deficits who repiv *ont the handicaps found in regular 
classrooms, particular!/ in rurul and urban-ghetto 
areas. Particularly ll [ active and genera 1 i zab 1 e 
programs and protocol materia!.' t ije i i e 1 d- tes t ed , 
packaged, and dirs.st mi nated under o her K due at i ona ]. 
Professions Dc j,/ U rrrui L -cf (i'Pijfd j.; ;<jj amb and related 
auspices . 

The trainees parti ei pa >‘c In f xl o lass room programs, 
which represent four d : :;t;,ct made Y. >* f earl/ childhood 
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learn to use new media and gjown teonnigues for 
effective Li a tiling of ulher-i in order to insure the 
multiplier effect of ti dining toad-as and teacher- 
trainers. The iM; rush ip ogpoi 1 vu 1 5 i os fm masters 
level students, in both devel opmonh and evaluation, 
are obvious. 
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of this HP DA m ' . v r am , staff guide the development of 
continuing inservicu training for professionals working 
with environmentally deprived and other educationally 
handicapped children. During the second year, ten 
teacher-cide teams receive three one-week massed-imput 
sessions woven into the thirty- three weeks of dis- 
tributed field practice. The intensive workshops 
familiarize participants with the unique training 
process and expose them to concert v a ued information 
regarding sociological, ant hr o^o ley ica 1 , medical, 
psycho 1 c j ic«il # ar .1 pcua jrgi ca 1 oop/’ ) dec a t i cm- necessary 
for prevent ing md/er da i i lay objectively 
developmental deviations in chd'i on. 

The f i re t year of field work i^ivi a sequence of 
sixty-four previously validated wiit^en and partially 
filmed learning opj sodo? corprj : ng sixteen training 
units plus the f i.o id - testing of eight new units* Tno 
trainees, working an their own sites* videotape their 
efforts t.o imp l client urn learning episodes (being 
critiqued by project stall) as well no record and 
critique similar of. forts of two additional to acne r- 
aide trainee teams; in each rorge-efivo satellite 
location. Continuing evaluation by progruov p mtici- 
pants, pro/ "t staff, and au i iiccpondcni cv..li* for 
influences annual piogr>ii i. c d j Me i Lions . bach subsequent 
year adds new groups rd J i M new- and new curriculum 
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for field testing and/or wide-spread dissemination. 
Provision is being made for up to ten quarter hours 
of credit for participants in this experience, 

Again, the opportunities f:>r fruitful internships 
in eva.l nation and development are apparent. 

A third program is just getting underway and the 
extent of the ramif ications of the program are thus 
not yet apparent. However, this third program, also 
funded under E'PDA, involves a group of students, 
many of them Chicano, in three distinct learning 
environments. The first environment occurs in 
acad c ic learning situations at the University of 
New Mexico in Albuquerque, through which the project 
is f unded. This training site also has responsibility 
for selection of the student population. The second 
learning environment will be provided by the John F. 
Kennedy Child Development Center and will consist 
primarily of block instruction in child development, 
both normal and abnormal, on individual learning 
disabilities, and on various types of developmental 
assessment and testing. The Chicano student trainees 
will t^en cjo to the Dos Angeles area for an extended 
live-in on-the-job training type of experience in 
Chicano neighborhoods. The possibilities of a worth- 
while internship in evaluation are currently indicated 
with th ; latter program. 
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It should be noted that che uniqueness of the programs 
at the John F. Kennedy Child Development Center makes 
them potentially viable in ways other than as internship 
sites. It is apparent that the various groups of 
JFK fellows and trainees might well profit from intensive 
training in either evaluation or development that could 
be incorporated into their other program activities. 

Staff vjould be on hand at -he university to make such 
instruction feasible. In certain eirc!in. stances It 
might even be possible tc cm 3 idor conducting a one, 
two, or three week intensive session for fellows operating 
at thn Kennedy Center plus opening the , to 

other persons involved in eai iy childhood education. 

1 0 ) Denvo r_ P ub l ie School s 

The Denver Public; Schools represent one of the tnree 
consortium agencies that are directly connected with 
the public; schools. Being a large district, Denver 
employs certain personnel who have evaluation and 
development responsibilities, and there are some 
indications that more personnel arc being employed in 
these genral areas. The working paper for the Denver 
Public Schools, Appendix II, describes quite well these 
various roles and the responsibilities there unto 
pertaining. The opportunities for internship, say or. 
the part of master. 1 * students, would Lo extensive. It 
should be noted that the student population of Denver 
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is representative of the student population found 
in most large urban areas. As such , opportunities 
for internship training on significant problems 
within American education arc multiple and obvious 
1 1 ) The Int er state K du cat_i on a l Re s our ce s _S er vice Cente r 
This eight state agency is housed within the Utah 
Department of Education in Salt lake City, 7 u, 
draws its funding primarily frem KCEA, Title V. 

Its major emphases ai e ussiul ir.g sen go j v in evaluation 
and planning. The state sope rin conic nt *5 of public 
instruction in the eight member states have designated 
evaluation and planning as hig' n pnority arc * r , 
Additionally, over thir ty behaviors of evaluators and 
planners have been identified as of immediate importance. 
Examination of the various behav ’ x O 1" & do signated as 
high priority by the Interstate Educational Resources 
Service Center j.nclmaces that they overlap considerably 
with many of the objectives established for this 
training program. mis being the case, it would seem 
that this consortium unc probably would operate 
primarily as a feeder .source, recruiting and sending 
to a training site g i c*u os o f public s choo l verso n ne l 
for intensive ^raining institutes in evaluation. It 
is also possible that the Service Center might be an 
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12) Northern Co lor ad c Ed ucat io nal Boa rd of Cooper ative 
Ser vices ( BOCES ) 

The Northern Colorado Educational Board of Cooperative 
Services has its main office in Br ' r , (Colorado, 
and provides direct service to sever, school districts 
in three counties. They are Boulder Valley, St. Varain, 
Westninis tor , Eastlake, Estes Park. Poudre, and Big 
Thompson. The three as c ociate nembers of the Educational 
Board v/ho coordinate activities with the seven partici- 
pating s choo 1 district s a r e Co 1 o r ad c State University 
at Fort Collins, The Driver si ty of Colorado at Bc.nld.--r, 
and the University of Northern Colorado at Greeley. 

The BOCUS program has three main elements; programs 
development, program evaluation, and information 
retrieval . Under program development the districts 
are given individual assistance in expanding existing 
programs and developing and implentei ing new urograms 
with long range, in termediate and sue v t-range gcals. 

Via 'he program of ovalution activitier, the districts 
arc aided in coordinating program development and 
measurement and in establishing on-going evaluation of 
programs thus developed. The information retrieval 
system initiates its operation witn educators rogue sting 
specific information. Then information specialists 
retrieve data by ar automated computer search of an 
educational data base consisting of ERIC (Research in 
Education) and CIJE (Current I nr He*: to Journals in Education). 
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Additionally, reference librarians are available who 
conduct manual searches for information relevant to 
individual requests. 

As is apparent from Mie v/orking paper for the 
Nw^thern Colorado Educational. Board of Cooperative 
Services (Appendix T} , unusual and dramatically 
feasible ideas for installation of an evaluative 
attitude within the school district are set forward. 
These ideas have been important determinants of much 
of the thinking and planning to date that ban gone on 
within the consortium. 

1 3 ) Colorado Depa r tme nt of Ed uca td on 

Located in Denver, the Colorado Department of 
Education provides a large number of varied intern- 
ship opportunities for masters level trainees in both 
evaluation and development. It should be noted that 
two of the proposed evaluation and development staff 
have been actively assisting the State Department of 
Education for the pact eighteen months to plan and 
conduct a unique state-wide assessment program; 
trainees will be able to interact with personnel on 
this project and intern on it. The n unbar of projects 
that the State Department initiates each year is large, 
and the opportunities for valuable internships would be 
great. Additional specifics about the organization 
of the State Department, particularly in regard to 
evaluation and development, are included in the working 
paper from that organization (Appendix (1). 




28 



n 



14 ) Ford Foundatio n 

Discussions arc currently underway with officials 
from the Ford Foundation exploring possible relation- 
ships between the Foundation and the Consortium. 

It is generally known that the Ford Foundation has 
been active in the field of education. For example, 
recently completed or in termination phases are o^er 
a score of programs operating nationwide in school 
districts, state departments, and universities. 

Many of these involve several districts or agencies, 
and generally are concerned with school improvement. 

The projects tend to be of a demonstration, rather 
than a research, nature, and great variety is apparent. 

Three Ford programs are of such a nature that they 
would offer possibilities for the placement of 
evaluation interns, if feasible relationships can 
be assured. One of these is a program to develop 
new school administration training programs. This 
has been conducted at five sites, and an evaluation 
intern might circuit-ride with the program director. 

A second program, titled Project Oppor tuni tv , 
operates out of Atlanta and involves trying to enrich 
the high school experience of many youngsters from 
the ghetto. It is a variation of the Upper Bound 
theme and involves special counselors in high school, 
college prep advisers, etc. 
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be able to use the services of an evaluation intern 
in evaluating this program. A third program on 
which an evaluation intern might work is termed the 
Leadership Development Program. Under this program, 
fellowships are provided annually for seventy non-urban 
educators and other potential community leaders. 

Those selected design their own training program 
and select their own training site(s). The three 
main groups pre *3ntly recruited into the Leadership 
Development Program are Chicanos from the Southwest, 
rural blacks from the South, and non-urban dwellers 
from New England; one of the Field Offices for this 
program is located in Denver. 

Even more unusual, and also under discussion with 
Ford officials, is the possibility of conducting an 
institute in evaluation for those of the seventy 
leadership development fellows v/ho felt that evaluation 
should bo part of their year’s training, Since 
fellows plan their annual programs in February and 
March, this would be an opportune time to get to 
them information on the possibility of an intensive 
training institute as one program option. 

1 5 ) U. S. Off i ce of Education Regional Office 

The Denver regional office of the U.F. office of 
Education has been helpful in the planning of the 
consortium. In addition to reacting to the proposed 
training program, it is also cuite possible that 
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certain trainees will profit substantially from an 
internship experience in the regional office. 
Evaluation is clearly an everyday responsibility in 
such a setting. 

It is apparent that the training program would be 
designed to take advantage of the unique and dynamic 
features of the various ayencics in the proposed 
consortium. It is likely that, with the passage of 
time and with the reality of operational funding, 
other organizations would be identified that have 
either unique internship opportunities or fh«^f have 
personnel who would profit from additional training 
in, evaluation and development. Two such groups 
that are under study are the National Assessment 
Regional Office, located in Denver, and the Belmont 
Project Branch, also located in this area. 
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III, R oles of Consortium Uni ts 

From the material presented above, it should be 
obvious that each of the consortium units possesses 
unique characteristics which serve as visible and pro- 
nounced justification for their inclusion in the consortium. 
It is anticipated that many other qualities and desirable 
features of the consortium units will become apparent 
after extended operational activities with them. In w?j / 
of specification and summary, the table on the following 
pages is presented to detail the extent of participation 
by the various agencies in the design phase, ana ch.- 
intended degree of participation in the operational 
phase. Additionally, roles of each institution during 
both the design phase and operational phase, am 
enumerated. Such a table serves to highlight the 
tremendous variety represented in the consortium units 
that make up the Colorado Center for Training in Educational 
Evaluation and Development. 
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' 1- Agencies in the consortium, their roles in designing, and their likely roles in inplenenting, the training program. 
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APPENDIX A 



WORKING PAPER ON THE EVALUATION , DEVELOPMENT 
AND DIFFUSION TRAINING DESIGN PROJECT 

William V. Mayer and James T. Robinson 

Biological Sciences Curriculum 
Study (BSCS) , Boulder, Colorado 

This paper will attempt to 'follow as closely as possible the 
suggested format for Working Papers of Consortium Constituent 
Agencies, although many of the questions are not germane to this 
organization. For example, we regard evaluation and diffusion 
functions as an essential part of development. Newly developed 
materials must be evaluated, and information regarding them 
must be disseminated. Development wi thorn evaluation and 
diffusion is a sterile task. 

There is a pronounced shortage of persons trained in evaluation, 
development anti diffusion. No collegiate programs and few 
organizations offer opportunities to develop these skills. A 
search for competent individuals, who have already developed 
skills in these areas, goes on con;: tartly. We have had 
considerable success in getting people with training and 
experience in thest three areas. They have been brought from 
all aver the United States after extensive searches and 
evaluation. Because we have concentrated on the few experienced 
personnel available in these fields, they have been able to 
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enter productive investigations within the Curriculum Study 
from their moment of hiring. We have not been able to afford 
the luxury of extensive training programs, although everyone 
learns from his on-the-job experience. 

A. The BSCS currently employs 12 persons at the Consultant- 
Administrative level. Of these, two have no degrees 
from institutions of higher learning, one has a bachelor 
of science degree, five have masters degrees, four 

have their Ph.D. Each has been hir3d for a particular 
skill level, and the background in which be had had 
extensive experience and training. 

By its very nature, the process of development also 
subsumes evaluation and diffusion. Thus, no one person 
is exclusively concerned with development, evaluation 
or diffusion, but in very large measures is concerned 
with all three. Each, however, probably spends more 
time in one of the three categories than the other two. 

If it is feasible to categorize the staff in terms of 
their major areas of interest, three would be concerned 
with diffusion, three with evaluation and six with 
develop me n t . 

B. Our current staff could stand to be doubled and still 
be kept very busy on its various tasks. Wc* consider, 
then, the need for three persons ecncerneJ wit n diffusiu* 
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three with evaluation and six more with development. 

They may be at any level, depending on the skills 
they bring to the organization. As indiaated by the 
backgrounds of our current staff, persons with no 
degrees and persons with PH.D degrees can contribute 
to this organization, 

C. If the organization's budget were to be increased 
by $100,000 annually, we could add approximately 
seven individuals to the staff, rather than the twelve 
needed, at salaries averaging approximately $15,000 per 
year, a minimal salary for highly qualified people. 

Under this provision, we would add two in diffusion, 
two in evaluation, and three in development at levels 
commensurate with their skills. 

D. The skills vary with the area in which individuals arc 
involved. For example, within our development function, 
there is a need for writers, artists, film makers, slide 
specialists, film strip developers, model makers and 
others who can contribute to a multi-media approach to 
education. In addition, of course, such individuals would 
have to be able to function within the program objectives 
as they make their contribution to a project. 

Evaluation personnel must understand design and method- 
ology of research, statistical analysis, measurement , 
and use of such instrumentation as the 
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computer . 



Personnel concerned with diffusion must know the 
characteristics of t ) population to which the materials 
are addressed, both student and teacher. They must 
be familar with administrative difficulties encountered 
in introducing new materials. They must be familar 
with marketing practices and teaching strategies. 

They must see how teacher behavior can be changed in 
such a manner as to make the new materials most effective. 

However, while some of the skills above can be taught, 
such items as imagination, creativity, and the ability 
to make intuitive leaps cannot. They must be attributes 
of all our personnel; those concerned with the develop- 
ment of materials as vv.ll as those concerned with evaluation 
and dissemination. 

II. Any person can profit by additional training; however, our 

personnel have been carefully selected and come to us already 
competent in the area in which they are to work. There would 
be little value in sending current personnel for specific 
training. Thus, the answers "A" through "H" below are 
hypothetical in relation to this organization. 

A. Some persons, who are currently within the organisation, 
might profit by training in computer technology or use 
of new and innovative instrumentation such as computer 
system instruction. However, at the present time, no 
individual is in need of training ir. the specialty 
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in which he is employed. 



B. Length of time would depend on the skill to be mastered. 
How long would it take someone to become familiar with 
computer-assisted instruction or making programs for 
same? How long would it take an individual knowing 
nothing of the computer to become skilled enough in 
its use? 



C. Were we to send individuals for any type of training, 
they would return to this organization in their former 
position, which is already graded at a level high enough 
to compensate for additional training . 

D. Salary, at least commensurate to what the individual 
is earning on the job, would be necessary to interest 
any of our personnel in specialized training. 

E. Wa could authorize leaves of absence, only if it were 
possible to "replace the person. r l:\is would mean that 
we would have no funds to contribute tc the individual 
while on leave, because we would be hiring an individual 
in his place, using the salary budgeted for the person 
leaving. For periods ol two weeks or less there would 
be no reason to hire a substitute or to curtail cue 
individual's salary. 
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F. If the current annual stipend is about $2/400 plus 
$500 per dependent, this organization could not afford 
the necessary supplement to provide additional training. 

A $2,400 sum is actually less than current graduate 
assistants get for half-time work within the University, 
and it is not practicable to assume that established 
investigators could subsist on such a stipend without 
adequate additional compensation, which we would be 
unable to provide for an extended period of time. 

G. At the present time, no personnel of this organization 
are in need of additional training, unless it were to 
be training of a very specific and somewhat esoteric 
type unlikely to be offered in a generalized program. 

H. As only three of our people have less than a master’s 
degree now, it could hardly result in an incentive for 
persons within this organization to take additional 
training, It might conceivably be an incentive to any 
new personnel to be added, but as I have stated above, 
we have never added personnel with the idea of training 
them on the job. We’ve added personnel already competent 
who can begin work on projects irr.med iate ly . 



III. The training components required ir. any such program depend 
largely upon the type of individual to be produced, and the 
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competence level at which he is to be considered ready. 

In terms of development activities, for example/ the following 
abilities would have to be considered: 

1. Demonstrated competency in an area of study that 
is related to the objectives of the organization, 

2. The ability to write at the level for which materials 
are intended. 

3. A knowledge of the population to whom materials are 
d i rected , 

4. An understanding of the capacity of teachers to 
adapt to new materials and ideas, 

5. An understanding of alternative routes of presenta- 
tion/ such as programmed materials/ etc. 

6. An understanding of the importa.ee of multi -sensory 
media involving the use of other than text pag£s, 

7. A knowledge of multi-media/ slide films, filmstrips, 
records, etc. 

8. Comprehension of a variety of teaching techniques 
such as team teaching/ audio, tutorial, mini lessons, 
etc. 

0. The ability to delineate specific objectives for 
materials development. 
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The ability to investigate, take the initiative, 
and learn as the objectives of the organization 
ovo lve . 

To evaluate newly developed materials, the individual would 
have to have a comprehension of evaluative techniques, know the 
limitations and the parameters of evaluation activities, be 
able to design an evaluation scheme, be able to ascertain the 
material's ability to reach its stated objectives, and be 
able to present evaluation data in such a manner as to be 
comprehensible to the population to whom it was directed. He 
also must be able to participate in materials development and 
contribute to the full cycle of the development task -- develop- 
ment, evaluation, dissemination. 

In terms of dissemination, comprehension of the materials produce 
with their evaluation is essential. Under standing school 
systems and their adoption procedures would be requisite. An 
understanding of the demands of the new mater als or curriculum 
upon teachers and students, and a facility for implementing 
a training program to meet these demands would be necessary. 

Certain of the above can be achieved in courses such as evalua- 
tive methodology. But the bulk of the materials can be achieved 
only throuyh a traineeship program wherein the individual is 
actually involved in developing materials, evaluating materials, 
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and disseminating information concerning them. No amount of 
coursevork substitutes for actual on-the-job training. 

Although some of the more fundamental and prosaic techniques 
can be developed in the classroom, only by writing does one 
learn how to write, only by evaluating does one learn how to 
evaluate, and only by disseminating does one develop the art 
of diffusion. 

here again, however, because new materials are being developed, 
to be evaluated and disseminated, there needs be a certain amount 
of creativity and imagination in the individuals picked for 
such a task. Nothing will be gained with developers simply 
reworking the already existing programs. Nev; ideas, new thrusts 
and directions, new packages of knowledge are required, and 
the recipient's development should not be stifled by prosaic 
and pedestrian coursework simply so he can do the work of the 
pas t . 

As stated above, our current staff requires no specific' training 
in program components because they have already dev loped the 
necessary skills. While they may be considered as instructional 
personnel themselves, there is very little advance that could 
be garnered for them by either formal coursework or internship 
oxperi once . 

The Biological Sciences Curriculum Study, an organization with 
over a decade of experience in development, evaluation, and 
dissemination activities, can well bo used both as 
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an instructional 



site and as an internship site for training developers, 
evaluators, and diffusion agents. However, this would be 
done in terms of actual programs currently underway. We would 
not consider structuring an ersatz superficial situation simulating 
the real thing. To participate most effectively in training, 
interns assigned to this organization would participate in 
actual development, evaluation, and diffusion activities 
under the guidance of experts currently employed by the Biological 
Sciences Curriculum Study. We would thus view an internship 
program as basically an on-the-job training activity in a real- 
life situation. 

A. All 12 consultant-administrative personnel on our staff 
would be willing to supervise c n internship in evaluation, 
development or diffusion. They would be willing to do 

so in connection with on-going actual programs, where 
the intern is taken into the organization and becomes 
part of it. He will be switched from task to task from 
time to time to give him the broadest possible experience. 

B. We believe that a one-year internship would not be too 
little for participating in the variety of developmental, 
evaluative, and dissemination activities of the organization, 
assuming that the intern has certain basic competencies 
with which to begin. Obviously, the more deficiencies 

the individual has prior to being enrolled in the program, 
the longer it will take to rectify then. 
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C. We believe that each agency in the consortium should 
participate in the selection of the students who will 
intern at that agency, 

D. As current contracts with funding agencies require the 
complete attention of each staff member, support for the 
intern and compensation for supervisory time would need 
to be provided through the consortium. An alternative 
that might be preferable weald include such funds within 
curriculum development contracts. 

E. Within the time commitments and funding of the actual 
programs currently underway, the BSCS is willing to 
accept a limited number of interns and involve them 
actively in the work of the organization. We do not 
believe that the establishment of artificial courses or 
disjunct dialogues would have half the value of the actual 
development of materials, evaluation of these materials, 
and dissemination of them. If, therefore, the BSCS 

can assist the evaluation, development, and diffusion 
training design project, it can best do so by involving 
interns in the programs in which it has the greatest 
amounts of expertise and experience. 
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APPENDIX L' 



WORKING PAPER ON THE EVALUATION, DEVELOPMENT 
AND DIFFUSION TRAINING DESIGN PROJECT 

William D. Romey 
and 

John F . Thompson 

Earth Science Educational Program 
(ESEP) , Boulder, Colorado 



I . 




Needs for Persons Trained i n Ev al uation, Development, 
a nd Diffusion in ESEP 

A. The project staff presently consists of 17 employees 

in residence at Boulder and 6 senioi college consultants 
retained on a part-time basis. Nearly all of the 
professional level personnel, 13 in number, are 
involved at various times in evaluation, development, 
and diffusion activities. However, certain staff 
members spend a relatively large proportion of 
their time in only one or two of these activities. 

The numbers given be lev? indicate the numbers of 
staff members who spend a significant portion, per- 
haps in excess of one-third, of their time on the 
activities specified: 

1. Evaluation - 7 people, including 1 with a Ph.D., 

3 with Master's Degrees, 2 with Bachelor's 
Degrees, and 1 with a High School Equivalency 
Certificate . 

2. Development - 7 people, including 1 with a 
Ph.D., 4 with Master's Degrees, 1 with a 
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Bachelor's Degree, and 1 with a High School 
Equivalency Certificate. 

3. Diffusion - 9 people, including 1 with a Ph.D. , 

6 with Master's Degrees, and 2 with Bachelor's 
Degrees . 

4. Editorial - 2 people, 1 with a Master's Degree 
and 1 with a Bachelor's Degree. 

5. Secretarial, Administrative, and Service 
Functions -8 people, including 6 with Bachelor's 
Degrees, 2 with college training less than the 
Bachelor 1 s level. 

Additional personnel needed: We could use additional 

people in all of these categories, and especially 
in evaluation, development, and diffusion. We may 
be hiring one additional secretary and one additional 
person whose time will be spent mainly in evaluation 
and development. Siueciali sts in overall materials 
design and layout and a computer specialist are needed. 
Effect of budget increase: If our organization's 

annual budget were increased by $100,000, and if 
this sum were to be used entirely for additional 
salaries, we might hire five or six additional 
people. One of these might be primarily an 
evaluation person (especially a computer specialist) , 
and the remaining staff members wculd be used in 
development and diffusion activities. Note however, 



in reality, that if our budget were increased by 
this sum, a quarter to a fifth of the new .roney 
would have to be used for support and publication 
activities. 

D. Specific skills needed: The evaluation person 

employed would need to have skills in design, 
cre&ti n of instruments, knowledge of existing 
instrumentation , research methodology, computer 
programming , and stettistical analysis. Most 
important of all, however, ho would need to be able 
to ask the right kinds of questions, to interpret 
results, and to write up his results in c arris 
understandable by laymen as well as fellow researchers 
Our group is especially interested in attitudinal 
analysis and he would have to be particularly 
well-prepared to deal with "dirty" systems consisting 
of many intertwined variables. Expertise in the 
area of helping devise self-evaluation systems is 
important . 

The people employed primarily for development 
purposes would need to have experience as school 
teachers, training in psychology, and preferably 
some experience as facilitator? in encounter groups. 
They would also need to be skillful writers and 
should have had a good deal of experience with 
resource materials useful in school situations. 

People whose primary function would be in diffusion 
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should have considerable experience giving oral 
presentations in public before groups of various 
sizes- They would also need to be highly skilled 
in writing and probably also in the development 
of graphic materials. 

II. Needs of Personnel Presently Employed by Our 
Organ i zation 

A. Our staff members in evaluation need additional 
training in how to deal with measurements in 
the affective domain and how to conduct 
longitudinal studies of various kinds. 

Associated research methodology and statistical 
analysis training are also needed. They need 
training in the design of instruments , data 
programming# technical writing# and interpretation 
skills for communicating with laymen. Staff 
members involved in development need further 
training in Low to operate ?ffectively as 
facilitators or trainers in encounter group 
situations. They also need training in writing 
skills and in the area of materials design. 

The diffusion staff needs training in the areas 
of graphic presentation and writing. 

B. We would be unable to release staff members for 
training on a full-time basis for periods in 
excess of one or two weeks. 
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Staff members 



could be given small amounts of release time 
for taking semester -long courses, however, as 
long as their effectiveness in their positions 
was not reduced. In general, most staff members 
would probably continue working full-time on 
their staff duties and would take any outside 
training on the side. Short, intensive training 
sessions followed by supervised work on actual 
studies related to our own project needs would 
be desirable. 

C. Persons with additional training might return 
to their former positions or to re-defined or 
new positions, depending on the particular 
skills they had learned. Promocions might be 
possible in the case of greatly increased 
skills. However, our budgetary limitations 
would be a controlling factor. 

D, Some of our staff members might feel that 
credits towards a Master's Degree and financial 
support to the extent of tuition remission plus 
a small additional stipend would be sufficient 
to interest them in further training. In some 
instances, it might be possible for employees 

to work for ESEP on a half-time or three-quarter- 
time basis and to spend the remaining portion 
of their time on outside training. An amount 
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of support equal to the amount of salary 
lost would be necessary for such people. The 
actual amount of financial support (or the amount 
of release time granted if there were a fixed 
stipend) could be adjusted as necessary. 

E. Our organization would probably not be in a 
position to grant extended periods of leave. 
Existence of grant-supported projects such as 
ours is tenuous to a large degree# e.g.,we 
could not guarantee the availability of a 
position after six months or a year. We might 
be in a position to grant leave without pay for 
periods of a few weeks or possibly for a summer 
session. Whether or not we could allow a leave 
of absence would depend# of course, on the 
w y ork load at any given time. The project might 
be in a position to supplement a trainee's 
stipend for a short period of time, providing 
the trainee would be willing to spend short 
periods of time back in the staff offices. 

F . We could not supplement a trainee's stipend 
under present funding. We could give small 
amounts of release time to our regular employees 
who might become trainees. 

G . mhere are four cr five people currently in our 
organization who would avail themselves of 
outside training. 
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If reasonable stipends were 



available, additional staff members might wish 
to avail themselves of additional training. At 
least two of our staff members are currently 
working toward Ph.D. degrees at the University 
of Colorado. These staff members, both working 
in the area of evaluation and development, might 
be interested in such training. If a good 
program in development were available, it is 
possible that several staff members might be 
interested . 

H. The possibility of earning an advanced degree 
would probably be a strong incentive L or people 
in our group. Many of our staff are already 
at the Master's degree level and credit toward 
a Ph.D. would be desirable. 

III. Internship Experiences a nd Seminar -T utorial Sessions 
Activities in this general area would best help our 
utaff members. Conventional coursework would be 
of little value. .Seminar- tutorial work in statistics, 
design of studies and instruments, affective areas 
of educatioii, unobtrusive measures, interpretation 
of results, and self-evaluation should he available. 
Supervised independent research or study projects 
wculd be of great worth. Peal studies on real 
problems could be undertaken via activities directly 
related to the operations of the curriculum projects. 
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X V , Supervised Internship Experiences and Participation 
in Actual Studies, "Real" Development and Diffusion 
P rojects 

Undoubtedly participation involving the ingredients 
above will be most important. "Mickey Mouse" 
components reflected in course credits, etc., are 
seen as having little value to our staff members. 
Tutorial sessions and frequent group meetings 
(in task-oriented encounter group settings) are 
seen as having great value. 

V , Use of E5KP as an Instructional and Inte rnsh i p oi t Q 

Our organization might be well used as an instructional 
and internship site for developers, encounter group 
trainers and facilitators, and for evaluators 
(especially in the area of attitudinal measures). 

A number of staff members presently employed by our 
group would be in a position to supervise internships 
in these areas. In general, our staff members would 
bo willing to undertake these functions. If there 
were any significant time involvement of our staff 
members, it would be highly desirable to have 
participating staff members (in faculty roles) placed 
on university appointments and to have appropriate 
sums of money paid to the project to reimburse 
us for time spent. If the staff member agreed to 
conduct training activities in addition to his 
normal project duties rather than in lieu of some 
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of them, such monies min t he paid directly to 
the staff member. In general, however, we would 
prefer to have such training activities conducted 
on project time and have any monies paid directly 
to the project. 
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APPENDIX C 



WORKING PAPER ON THE EVALUATION, DEVELOPMENT 
AND DIFFUSION TRAINING DESIGN PROJECT 

James C. Moore 

Southwestern Cooperative Educational Laboratory 
(SWCEL) , Albuquerque, New Mexico 

The purpose of this working paper is to provide some 
considerations for the design being developed by the 
consortium. The paper will not react to each of the 
specific points in the suggested working paper format. 

However, most cements will be related in part to Lhw 
format, and some additional thoughts will be presented. 

It can be pointed out initially that this organization 
has needs in the areas of evaluation, development and 
diffusion. This certainly is not a unique need among 
regional laboratories or similar organizations. At the 
same time, however, we would probably have to say that we 
are "fully" staffed. That is, we do not have a poo' of 
money standing by to be used to hire evaluators, dovel >pers 
and diffusers even if they were available. This is not 
unique, either. Another reality is that nearly everyone here, 
at what might be called the professional level, is doing 
things that could be identified (quite easily by most 
external observers) as evaluation and development. 
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Given this kind of situation, our inclination is toward 
a training program directed at employed people who already 
have significant job responsibilities in the area of 
development and evaluation. This kind of approach would 
have as its focus striving for significance by ameliorating 
the development and evaluation needs of people already 
in positions, rather than recruiting new people into open 
positions. Hiring additional staff may become more 
pertinent when we open many "new” positions. This is not 
likely to occur in the next year or two. 

Thus, we would suggest a new title for the Consortium 
(or at least an additional focus) : 

A Consortium Designed to Improve the Skills of 
Educational Developers and Evaluators (or something like this) 

Given the implications of the title above, what would 
be some of the important points to consider? 

1. Could the funding agency (ies) be convinced that 
perhaps a more crucial need is to improve skills 
of currently employed personnel than to recruit 
new people into training programs? I think a good 
case can be raade. Some of the following ideas 
could be expanded on in making the argument: 
a. The population of interest can be readily 
identified, i.e., almost every educational 
organization has personnel with significant 
development a 1/e valuat ional respons ib il i ties 

who lack significant and requisite skills. 
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b. The "traditional" programs in training educational 
researchers (with a few exceptions) have not had 
significant impact, although many highly qualified 
people have completed the programs. Why, in 
essence, design the same process, call it the 
training of developers and evaluators, and 

throw in a little internship to boot? 

c. Many persons feel that a more successful training 
model has been developed via the AERA presessions 
which have been aimed at skill improvement for 
already employed personnel. To the extent that 
this is true, imagine what might be done with 
the AERA presession model if sessions were 
designed to last a month, six months, or a year. 

d. In amplification of "a" above, it should be noted 
that many organizations also have competent 
personnel who could and would contribute to 

the program if the resultant effect was the 
improvement of the people working with or 
for them. 

e. Through a consortium arrangement, bolstered by 
other cooperating organizations, a network of 
experiences (i.e., internships) could be provided 
for the participant which relate directly to 

his job. (Conceptually, in most traditional 
programs, internships are designed around 
experiences a person might hold at a future date, 
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whereas a strong point could be made for 
internships conducted concurrent with employ- 
ment and concordant with present job responsi- 
bilities. For example, in a not- so-recent 
issue of the Harvard Educa ti onal Review , it 
was noted that practicum teachers reported 
deficient skills and know ledge in the area of 
tests and measurement as a high priority problem. 

On the other hand, pre-service education students 
gave such courses low priority. It is suspected 
that many people now in evaluation and development 
roles with specific r is yonsibi lities feel "needs ' 1 
in multiple skill areas.) 

f . The program could be given the flavor of individually 
prescribed experiences and training. 

g. Although evidence, either supporting or refuting, 
is lacking , it would seem logical that a program 
that improves current personnel is considerably 
more cost-effective than a program which calls 
for making positions available for new personnel, 
and for training new entrants into the field. 

Other considerations related to the consortium training 

prog ram . 

a. The program should be designed to relate to the 

individual's day to day responsibilities, regardless 
of where the individual might be a participant. 
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To clarify this a bit . , . the idea of "leave 



of absence/' or culminating in a "master's degree/' 
etc,, possibly should not be made explicit. It 
is quite conceivable that a participant might be away 
from the home organization working (interning) , 
varying from taking classes at the Laboratory of 
Educational Research to spending a period of time 
being a teacher aide . . . the idea being that 

the participant really never leaves his day to 
day responsibilities. 

The above approach in "a” might also ho more 
easily accepted by a participating organization. 

For example, if we participated in a program which 
required a leave of absence from our organization 
for a period of time, say a semester or two, and 
the individual would come back to u u after the 
leave with new skills, etc. , we would probably 
participate. However, the individual sent to the 
training would be someone we could "spare" for a 
length of time and not, in all likelihood, be 
the individual who would gain the most, since an 
entire program effort might have, to be delayed. 
Obviously, a major task would be how to design 
and manage such a consortium. The Laboratory of 
Educational Research would have the overall task 
of implementation and management. The substance 
of the original proposal would not change since 
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the thrust would be toward development and evaluation 
with interdisciplinary overtones. 

Finally, I should like to indicate that this laboratory 
is quite interested in actively participating in the continued 
design, and hopefully the future implementation, of the 
consortium, regardless of the acceptance or rejection of the 
ideas expressed in this paper. Although actual commitments of 
time, personnel, or money, require further deliberation, the 
probability of full participation is quite high. Thus, this 
organization could be considered not only as a source for program 
participants, but concurrently as a training site. 

In conclusion, I have not responded to the specifics asked 
for in the suggested format for working papers since it was 
indicated at our recent meeting that your original approach 
would likely be modified greatly on the basis of data inputs. 

Should you need the information asked for in the format under 
IA, IB, and IC, I will obtain it. I A would take a day or two 
to "decide”, while IB and IC wouldn't be much more than off 
the top of the head reactions unless we take more time to study 
the situation. 

ID and IIA, I feel, should be part of the consortium 
process. That is, one of the major objectives of the consortium 
would be to identify the skills needed by individuals. Other" 
wise, all I can say is ‘'yes" , all those things under "ID and IIA" 
are needed, and probably more. It would seem that in the 
proposal a scheme (process) could be presented that would be 
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used to identify the specific skills as part of the training 
experience. This would further the idea of aiming at individual 
needs and not pre-determining the things we think are necessarily 
important, {Things like design, measurement, and statistics 
can be given as pr obable needs, however, in a proposal.) 

I can be more specific on the things under II, III, and IV 
when it is decided on the approach you are going to take in 
regard to trainee population, I realize that you too are holding 
your approach strategy until you get some feedback on what 
we tend to like. I have already indicated this organization’s 
interest in participating, as asked for in V. 

I will be most happy to clarify or expand aspects of this 
paper if requested and hope to contribute to the continued 
design of the consortium. 
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APPENDIX D 



WORKING PAPER ON THE EVALUATION , DEVELOPMENT 
AND DIFFUSION TRAINING DESIGN PROJECT 

Masahito Okada 
Member of Professional Staff 

Southwest Regional Laboratory for Educational 
Research and Development (SWRL) , Inglewood, California 

To provide answers corresponding to the questions in 
the proposed format extensive deliberations would be 
required at this organization. Therefore, many of the 
questions will remain unanswered or answered in a rather 
imprecise manner. Should you require more information or 
exact data, it could be provided for you at a later date. 

I. A. SWRL employs approximately 180 employees. The 

professional staff are approximately equally divided 
at the doctoral, masters, and bachelors degree 
levels. The evaluation staff is concentrated large- 
ly in the integration division whose function is that 
of "diffusion." A few staff members at the bachelors, 
masters, and doctoral levels are involved in the 
evaluation of new projects produced in product 
development. It is their function to provide for 
the formative evaluation of new products. There 
is no separate group within the organization whose 
primary function is that of evaluation. Approximately 
35 persons are involved directly in the development 
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of new products. Approximately 50 persons are 
involved in the diffusion task. The distribution 
of academic degrees in both divisions approximates 
that found in the laboratory as a whole. 

B. SWRL has recently been funded to build a plant 
capable of housing approximately 430 employees. 

The building is scheduled for completion in the 
summer of 1972. At that time it is expected that 
there will be a substantial increase in new personnel. 

I. C and D , and II , III, and IV are not responded to 
directly, but it is apparent that the increase in 
personnel will create a large demand for new entrants 
into the fields of evaluation and diffusion. 

V. SWRL has an established internship program in exist- 
ence under the direction of the Division of Resource 
Services. As indicated by our Director, trainees in 
the program would be eligible for these internship 
slots. This internship arrangement is more fully 
explained in the internship guidelines which are en- 
closed . 

Skills and Knowledges Desirable in Product Development Personnel 
Most of the following apply to the activity heads who 
are directly responsible for the development of new programs. 

In some cases these same competencies can be seen to be 
desirable characteristics of all personnel working in product 
development . 

1) A substantive knowledge of the area (reading, music, 
etc. ) 



65 



2 ) 

3 ) 

4 ) 

5 ) 

6 ) 

7 ) 



8 ) 

9 ) 

10 ) 

ID 

12 ) 

13 ) 



14 ) 



At SWRL, an understanding of the criterion refer- 
enced approach to product development. 

An understanding of formative and summative evalua- 
tion procedures. 

The ability to design appropriate instruments to 
evaluate developing programs. 

A basic knowledge of research strategies and statistics 
The ability to write for various audiences (children, 
teachers, etc.). 

The ability to administrate and supervise in order 
to capitalize on the potentials of all co-workers 
in a joint venture. 

A basic understanding of educational psychology 
and learning theory* 

An understanding of child development, specifically 
as it relates to product development. 

Some appreciation for the realities of classroom 
management . 

Divergent and convergent thinking abilities. 

The ability to search out and evaluate relevant 
research studies. 

The ability to use business managements in product 
development . 

A knowledge of production logistics and variables 
(e.g., cost, types of art, formating, media, etc.) 

A knowledge of consumei characteristics (e.g., how 
much will a schoo] district pay for a new art program?) 
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15 ) 



16) A knowledge of typical curricular structuring in 
the public schools (e.g., how much time is usually 
allotted to reading in grades K-6?) . 

17) A knowledge of current innovations in education 
which may allow for new programs (e.g., open 
space programs, team teaching, television, com- 
puter assisted instruction, etc,). 

18) Competence in measurement procedures. 
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SWRL Inter ns hip Guideline 



Because of the obvious relevance of the activities of 
SWRL to certain college and university graduate level 
programs, the following guidelines have been developed to 
assist in the establishment of internship arrangements. 

1. It is expected that whatever the nature of the 
practicum experience, it be mutually beneficial to 
the trainee and to SWRL. In other words, it is 
anticipated that the intern v/ill profit from assisting 
in the attainment of a SWRL project goal. 

2. Internship arrangements will generally be set up 
on a part-time hourly basis rather than full time. 

Because of the considerable variability in class 
schedules of graduate students, it is appropriate, 
insofar as it does not hinder SWRL opeartions, to 
tailor the work-week commitments to the intern 's 
schedule. Sometimes, for example, the intern 

would devote 5-10 hours per week to bis SWRL project. 

On other occasions, particularly during summer and 
other academic vacations, almost a full-time schedule 
might result. Although some variability in the 
work-week schedule can be arranged, the average 
should be a minimum of 10 hours per week. 

3. Because : ae intern will either receive academic 
credit (e.g., through special studies courses) or 
generally beneficial experiences during the internship, 
regular salary arrange nents are inappropriate. 

However, in order to compensate partially for the 
inconvenience oi \ travel requirements associated with 
off-campus internship activities, the intern will 
receive $2.50 per hour for the time he devotes to 

the SWRL practicum experiences. In light of the 
intern's usual level of training when he commences 
the practicum (usually the equivalent or a master's 
degree plus additional training in product developmei t 
activities) , this represents an hourly rate approximately 
half that which might be earned by comparably trained 
personne 1 . 

4. In the case of f ^ach internship arrangement, the 
SWRL Director or his delegated representative would, 
in conjunction with the intern's faculty advisor, 
work out and state briefly in writing the nature 

of the intended internship experience. Both the 
representatives of SWRL and of the participating 
university would approve this written statement. 



5. Supervision a ivi v aluation of the intern's 
activities will be conducted jointly by the trainee's 
immediate SWRL superior and by a representative of 
the participating university. 

6. As part time staff personnel, all hiring, financial 
arrangements, etc. will be handled through normal 

SWRL channels. 
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APPENDIX E 



WORKING PAPER ON THE EVALUATION , DEVELOPMENT 
AND DIFFUSION TRAINING DESIGN PROJECT 



Barry B. Beal 
and 

Gerald E. Elledge 



Denver Public Schools/ Denver, Colorado 



Enclosed is a response to the working paper request, 

eral general statements have bearing upon our partici- 
pation : 



The Superintendent has approved our part icipation 
in the design of this project. In the event 
your program becomes operational and cur continued 
participation is desired, he indicated that such 
participation would be subject to review. 

Our proposed 1971 Budget is an austere budget 
and supportive funds are not available. 

Sabbatical leave requests and inservice fund 
allocations would c mpete vitn other such requests. 

An intern# on-the-job training model, has the 
most appeal. However, one semester or full year 
sabbaticals may be possible. The Executive 
Director of Planning, Research, and Budgeting# 
has indicated approval of this idea and has 
suggested that one or two persons night participate. 
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The potential benefits of such a consortium- 
type training program are recognized* 

It is probable that where mutual benefits are 
evident that cooperative working relationships 
between the consortium and specific offices or 
department? of the District may be arranged but 
subject to necessary adminis trative approval. 

A description of likely participants includes: 

- persons having an M.A. degree. 

- those currently involved to some degree 

in evaluation or program development. 

- those with six or more years* experience 

in the Denver Public Schools. 

Office of Planning, Research, an d Bridget ing . 

Within the Denver Public Schools, there are certain office 
persons trained in evaluation, development, and diffusion 
who would be of benefit to the District. The principal 
area of demand is found within the Office of Planning, 
Research, and Budgeting. A relatively small pool of 
potential trainees exists here, however* There are three 
Ph.C. and five master degree level people in this area. 

A course of study for these people might include research 
design and methodology, statistical analysis, objectives 
writing and analysis, sampling procedures, and change 
stategies. 

ERIC 
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It would be difficult tor these people to take a year’s 



leave for this purpose as it would mean replacing them 
with inexperienced people in the Research Department. 
Therefore , the best training model from the Research 
Department's viewpoint would be one which incorporates 
some type of on-the-job upgrading. This might take the 
form of three half-days, plus Saturdays or evenings, on 
campus, and the rest of the time on the job for a period 
of one year, or an intensive summer program with periodic 
meetings throughout the academic year, The inducement 
for participation for this group of people would be in 
graduate credits for salary increases and stipends for 
Saturdays, evenings, or summer sessions, A minimum of 
five people in the Research Department would be interested 
in participating in such a program. The Research Department 
might be used in part for training interns in evaluation, 
there are people in the department who would be willing to 
supervise such a program. 



Evaluators in School Settings. 

Another area in which persons trained In evaluation would 
be of benefit is the group of teachers resposible for 
evaluations within school units - approximately twenty-five 
in total. These people arc responsible for the testing in 
each of their secondary schools. This role could be expanded 
to one of curricular evaluation and development as veil as 
testing. A potential for ninety-one evaluation people 
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exists in the elementary schools. No such position exists 
now in the elementary schools and probably will not in 
the near future. However, this function is being fulfilled 
as an additional teacher responsibility. These persons 
then, could serve as a pool of potential trainees. A 
course of study for these teachers might well include: 
measurement, instrumentation, objectives writing and 
analysis, some research design, and statistical analysis 
and change strategies. 

Persons in these roles could spend a year on campus in intensive 
study. It is possible for some teachers in the Denver 
Public Schools to obtain a sabbatical leave of absence for 
one year at one-half salary. This financial arrangement, 
along with the usual trainee's stipend, would probably be 
sufficient to attract our teachers. In addition, they 
would be encouraged to participate in a program of this 
nature if they could earn an M.A. degree or academic credits 
for salary purposes beyond the M.A. 

Prog ram Specific Internship s. 

Another possibility would be to identify specific program 
areas for evaluation, development, and diffusion such as: 

. Guaranteed Performance Contracting 
. State Assessment and Accreditation Programs 
. HSEA-Titled Programs 

. School Unit Analysis (ala IDEA Schools) or Decen trali za tion 
. Reading Research Evaluation 
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The topics uoove are likely concerns of high public 
interest during the Decade of the 70 E s and school 
districts will, of necessity, be required to pursue 
evaluation and development programs in these areas. 
Persons for one or more of the above areas could be 
identified to participate in a cooperative internship 
with the consortium. It is likely that a full year 
or a semester sabbatical leave could be granted to 
qualif ied individuals, j This person (s) would, prior 
to the internship, be charged with the development of 
a viable evaluation, development, and diffusion program 
for the Denver Public Schools ii. a specified area. 

There would be a potential cf then giving such an 
individual a special assignment to implement the 
program design developed during the internship. 

The course of study for these people would include 
research design and methodology, instrumentation, 
change strategies, data interpretation, objectives 
development, and measurement . It would seem that this 
approach might allow national appeal but with a focus 
and flexibility which could serve highly specialized 
needs. One person might be des ign ated for perhaps 
each of two program areas. Such individuals, most 
likely, would have a masters degree, but would be 
interested in receiving an Educational Specialist 
and academic credits. 
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APPENDIX F 



WORKING PAPER ON THE EVALUATION, DEVELOPMENT 
AND DIFFUSION TRAINING DESIGN PROJECT 

Douglas Sjogren 

Northern Colorado Educational Board of 
Cooperative Services, Boulder, Colorado 

The Northern Colorado BOCES is an organization 
similar to what are* often referred to as immediate districts 
This board serves seven school districts in Northern 
Colorado. These districts have one-fifth of the public 
school teachers and students in Colorado. This particular 
board was established to work on diffusion, development, 
and evaluation of educational programs in the seven district 
This is not a typical function of such boards, but we 
believe it is a viable and realistic kind of function for 
such an organization. In effect, we are testing a model 
for this function with our board. This brief background 
was included to provide a perspective for our points-of- 
view. 

Before responding specifically to the questions in 
the suggested format, 1 would like to present some thoughts 
I have with respect to evaluator training. I have serious 
doubts about the success of any evaluator training program 
if it is designed to produce "evaluators' 1 . It seems to m a 
that evaluation of educational programs is a process that 
involves all. components of the system. Whether or not 
meaningful evaluation takes place in a public school is 
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not nearly so dependent on the school having a "researcher 1 ’ 
or "evaluator" as on whether the staff, at all levels, is 
oriented to evaluation. This statement is conjecture on 
ray part based on rather limited experience along with the 
observation that school systems and educational programs 
that have their "evaluator" do not sta, t-out in terms of 
effectiveness or impact. Today, most of the real evaluation 
(i.e, that evaluation that makes a difference) in the schools 
is done by the staff. It seems to me our task is to 
strengthen this evaluative effort which is being done by 
involvement with the staff rather than by training a 
specialist who typically remains on the periphery of the 
system. Certainly people with special skills are needed 
in order to improve the evaluative effor-t. The range of 
skills is large, however, ar.d it is unlikely that all can 
be developed in one person. It is more likely that many 
of the skills can be developed by recognizing and capitalizing 
on the range of interests and abilities among the staff 
members. I guess I am saying that if one person is to be 
trained for the evaluation role in a public school he 
shouLd probably b a rather high-level administrator who 
has decision-making responsibility at the highest level. 

This person should be a generalist who can establish in- 
service programs and use other techniques to develop an 
evaluative point-of-vinw in the staff. He should be awc*re 
of the many kinds of information gathering procedures ar.d 
should establish procedures for developing competence in 



these areas in his staff or know where the competencies 



can be obtained. He should also be able to establish a 



system for gathering, analyzing, and synthesizing the 
information as an on-going system. 

I have difficulty distinguishing evaluation from 
development in a school context so that these remarks 
apply to development as well . With respect to diffusion, 

I guess I feel that diffusion will take care of itself 
if the schools become developmentally or evaluation oriented. 
As the schools become able to examine and question existing 
programs they will be more likely to search for alternatives. 
There are some specific skills that are important in 
diffusion, however. One is to know where to find alternatives 
and to evaluate them. Another is the important skill of 
preparing diffusion materials that are useful for the 
potential user- 

The evaluation and development effort of the Northern 
Colorado BOCES has two general aims or functions. One is 
a training function in which we are striving to develop 
in the staffs of the schools; an awareness of the need 
for systematic development and evaluation in their districts. 
The second function is to provide for the districts certain 
competencies in evaluation and development that they do not 
have in the district and which any one district would not 
be likely to obtain. We also have immediate access to 
the ERIC data base. 
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The reader should be aware that our operation started 



I . 



on July 1. Consequen cly , many of the answers to the 
questions are just "best guesses' 1 about our or a similar 
organization . 

A. Evaluation: - 3 F.T.E. 



One 


Ed . D . 


- full 


time 


One 


Mas ter s 


- half 


time 


One 


G. A. 


- half 


time 


One 


Secretary 


- fuJl 


t ime 



Development - 3 F.T.E* 

One Ed. D. 

One Masters 

One Secretary 

Diffusion (Retrieval Center) - 12 F.T.E. 

Three Masters 

Three Bachelors 

Three Teonical 

Three Secretarial and Clerical 

B. At the present time, we do not need additional 
people. I would expect that we will want at least 
one more full-time person at the Masters level in 
both evaluation and development. We expect to use 
many temporary consultants in these areas. 

C. A budget increase of $100,000 in our organization 
would be tied to a specific program, and I can not 
indicate how it would be used except as I have 
indicated in B. 

D. Specific skills needed in evaluation and development 
component. All of those listed in the format are 
needed plus par t icipan t-obser vor skills, administrative 



o 

ERIC 



78 



skills , economic analysis, political sensitivity, 
and human relations. 



II. 



A. Within the evaluation component v-j are weak in the 
areas of participant-observation, administration, and 
economic analysis. 

B. In our situation, short term, concentrated kinds of 
training sessions would be pro te/ rod. 

C. Most would stay in their prejrn.i position. 

D. Financial support would not be needed in most situations. 

E. We would probably not grant leaves of absence. We 
would hire a person with certain skills to do certain 
things and then use on-the-job and in-service training 
to develop both depth and comprehensiveness. 

F. Short-term training sessions would be regarded as 
part of the person's job. 

G. I think each of us in evaluation miyhc avail ourselves 
of certain kinds of short-term training. 

H. Net for the persons presently employed. 



III. I feel internship type experiences are very important for 

training new people who will serve as directors of evaluation. 



IV. In working with persons already employed, I feel that short- 
term intensive kinds of training are effective. This training 
is most effective, however, if the training agency provides 
follow-up support and consultation as the person might start 
activities after the training session. 
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V. 



I feel our organization can be a useful training site for 
interns who would be interested in working in an organization 
similar to ours or in a school district. We can provide a 
variety of experiences for the person in the three areas of 
evaluation, development, and diffusion. We have four 
persons in the organization who could supervise an intern, 
and all would be willing to do so. 
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APPENDIX G 



WORKING PAPER ON THE EVALUATION, DEVELOPMENT 
AND DIFFUSION TRAINING DESIGN PROJECT 

John Ahleni.us 

Colorado Department of Education (CDE) , Denver, Colorado 
. Personnel Needs 

A. Presently, the CDE employes seven professionals in 
evaluation, four of whom have Doctorates* One 
employee (Masters plus' has diffusion as a role* 
Nineteen, one-third of w*hom have earned their 
Doctorate, are in the Improved Learning Unit, which 
is the closest thing we have to your definition 

of development. 

B. We were recently reorganized and are currently 
properly staffed or nearly so. 

C. If the Evaluation Unit, gets a requested $115,000 
supplemental budget, we will commission $62,000 
for professional and support personnel. These 
additional personnel would most likely be Bachelor 
or Master-level trained. We would not need 
additional Doctor al- level personnel. We could use 
para-professional help in certain aspects of our 

data processing. With additional money in development 
and/or diffusion, the CDE w'ould likely place more 
emphasis on diffusion. Assuming ve have the 
curriculum areas covered, we need help in carrying 



the message to the teachers of the state. For 
example, one charge has been to reach 5,000 teachers 
with the 11 Invitation to Improved Learning" package 



in one year. 

D. In evaluation, we need help in item writing and 
field testing for the item pool in our state 
assessment instruments. Detailed efforts are needed 
in the "nuts and bolts" area of checking congruence 
between goals-ob ject ives-items . We need help in 
communication with 'hardware" people and instrumenta- 
tion for processing judgmental data. We should 
have a para-professional trained to use randomizing 
techniques for many studies. 



In development and diffusion , we probably need the 
skills (roles) that Ronald Havlock elaborates as 
needed by change agents: 

1. Development of a viable relationship. 

2. Assessment of needs (diagnosis of the problem) 

3. Acquisition of relevant resources. 

4. Selection of an appropriate solution. 

5. Achievement of acceptance. 

6. Stabilization and se ] f -renewal . 

This appears to be salesmanship, but it is supposedly 
more than that. Our personnel need training sessions 



Tlavlock , iTTcT; A guide to innovation in education . 

Ann Arbor, Michigan: Center for Research on Utilizat ion 

of Scientific Knowledge, The University of Michigan, 1970, 



o 

ERIC 



82 



(simulation possibly) on these skills. Some' of 
Havlock r s skills (roles) might be untru Lnuble . 

Is it possible that one must be "born" with the 
ability to establish a relationship? 

II. Training Needs 

A. The training needs described in this working paper 

predominantly represent on-the-job types of training. 
Although several in evaluation want additional 
training in design, research methodology, statistical 
analysis, objectives analysis, measurement, survey 
research, etc., it is assumed that they brought to 
the Department these skills as a result oi our 
pre-service training. In this case, however, the 
ideal is not always congruent with the real. 

We need a model, and training in that model, for 
inservice -training of local education agency personnel 
in evaluation. Our unit with consultants from the 
Title I department are called upon to "teach" LEA 
representative how to evaluate their programs. 

We need training in instrument construction for 
assessment in the affective domain. Again, Title I 
schools repeatedly ask for this kind of help. The 
proposed training project might even go directly 
(i.e., bypass the CDE) to the LEA Title I directors 
in the regard. 
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There are eignt Colorado schools piloting the PCI 
(pupil centered instrument) from the Belmont Group, 

An additional 47 Colorado districts are involved 
with the CPIR ( Consolidated Program Information 
Report) . These two groups are potential audiences 
for training in evaluation, 

B. The Colorado Department of Education endorses and 
would coordinate more fluidly with intermittent 
on-the-job training, it currently is allowing 
15 employees two weeks of training (4 days in 
training, few months back on the job, 2 days in 
training, a month on the job, and a final 4 days 
in training) , This seems to be highly feasible 
and practical as one's work waits for him while he 
is away on any kind of leave, 

Ct In most cases, the department views on-the-job 
training as upgrading for the employee f s current 
position. Following the recent reorganization, 
many people found themselves in redefined positions. 
That, however, was not due to any training that 
they received. 

D. Financial aspects of r.hort duration training programs 
are not critical for CDE personnel. Colorado 
Department of Education employee's accrue 2 weeks 
of educational leave with pay per year. Training 
sessions that pay stipends (and some have) are, 
therefore, considered as supplements to income. 
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E. Year leaves or semester leaves arc? grunted, but 
not in large numbers. As mentioned above, one to 
two week training sessions have been viewed most 
favorably by the Administrative Council with some 
four week and fewer six week sessions getting the 
nod of approval. 

F. See D above. 

G. About two-thirds of the evaluation staff would use 
short duration training courses in evaluation. 

Most of the nineteen in development would partici- 
pate if the right kinds of training are planned. 

(The matter of definition of development and the 
Depar traen t 1 s participation needs to be negotiated 
with Dr. Chris Pipho and his staff) . In a study 
conducted within the department a few years ago, 
seventy percent of all employees listed diffusion 
as one of the roles they performed, in carrying 
out their jobs. Fifty percent listed it in the 
top five of their listed roles. 

H. Earning a Master's degree would not be an incentive 
foi training. Most professional employees have at 
least a Masters. The Colorado Department of Education 
is most in need of short term training programs. 

111. This paper does not address itself well to the pre-service 
training of employees. Generally, experience with the 
tasks assigned to the particular job within the Department 
is the type of training perferred over course work or 
an internship experience. 
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IV. Concentrated courses coupled with simulated or "on-the- 
job 1 ' experiences best utilize the time available for 
training present employees. For instance, the computer 
printout of the recently completed pilot for state 
assessment could serve as the text for a course in 
data analysis and reporting. 

V. The Colorado Department of Education would readily 
agree to be an intern site in all three areas. Not 
only would we profit from the skilled labor, but it 
v/ould provide us with a pool of future employees. 

We are not in the position of being able to contractually 
provide jobs for individuals assigned to us for an 
internship experience, but departmental turnover is 
high enough to provide some promise for employment 
of capable and well-trained individuals. Generally 
speaking, the Department needs personnel who have had 
public school teaching experience. 
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